This paper examines the effects of survey mode patterns of survey response, paying special attention to the conditions under which mode effects are more or less consequential.
We use the Youth Participatory Politics survey, a study administered to 2,920 young people either online or over the phone. An important feature of this survey was that some of the respondents were assigned to mode, while others chose the mode in which they completed the survey. This feature allows us to account for the extent to which potential differences across mode are confounded by selection effects. Our results provide consistent evidence of mode effects. The internet sample exhibits higher rates of item non-response and “no opinion” responses, and considerably lower levels of differentiation in the use of rating scales. These differences remain even after accounting for how respondents selected the mode of survey administration. We demonstrate the substantive implications of mode effects in the context of political knowledge and racial attitudes. Internet samples are likely to appear less politically knowledgeable, possibly due to decreased engagement with the survey and/or greater use of satisficing strategies. Internet respondents also reported lower levels of interracial social interaction, which may indicate that the anonymity associated with internet surveys may reduce biases associated with social desirability effects. We conclude by discussing the implications of our results for comparing data obtained from surveys conducted with different modes, and for the design and analysis of multi-mode surveys.